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ABSTRACT 

Teacher and administrator feedback about a study on 
computer-assisted instruction (CAI) is described in this report, The 
hypotheses, methodology, and conclusions of the overall program are 
briefly discussed as they pertain to the feedback aspect, (The CAI 
program dealt with the effects of choice situations upon the 
students 1 engagement in learning. Subjects were chosen from Grade 4 
and 5 classes in a low income area elementary school.) Since the 
experiment depended on evaluation by observation, the researchers 
felt it necessary to have a good operational rapport with the school 
personnel, The teachers provided information that verified some of 
the researchers 1 expectations concerning the effect of social 
determinants on choice behavior- This report was compiled from 
information collected by questionnaire and discussion* (MC) 
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This paper describes the results of a research feedback session 
held with some teachers and administrators from Mayfair elementary school 
which is located in a low-income area of San Jose, California, This 
meeting occurred at the Stanford Center for Research and Development in 
Teaching approximately six months after the completion of the first phase 
of our CAI project, The Mayfair school staff was invited to partici- 
pate In this meeting in order to: discuss any problems created in the 
school by our research activities, hear about the results of the project, 
offer their interpretations of the data and make suggestions about directions 
for future research* 

The primary objective of this research was to investigate the effects 

of choice situations upon students ! engagement in learning*"'" We uti- 
lized a Computer Assisted Instruction program to study choice situations 

which occurred while the children worked arithmetic drill exercises, and 
evaluated their engagement levels with an observational scale that 
measured how intensively they attended to the learning task. The child- 
ren could work on the CAI problems during ten one^haif hour daily sessions. 
They could also stop working after every five minute block of problems 
was presented during each session or completely refuse to work on the 
computer any time within the ten day period. The subjects were randomly 
selected from four fifth grade classes and one fourth grade class at 
Mayfair school and placed in either Choice or Non-choice conditions: 
(1) The Choice condition (N=30) gave them the option of selecting pro- 



See Appendix A for a more detailed description of the research 
design. 
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blems that were ''harder 5 ' "easier 15 , or of the "sama 5 * difficulty levels 
than they worked during the previous five minute period, (2) and the 
Non-choice condition (N^21) was fixed at approximately the 70% correct 
level during most of their time on the computer. Continuous observations 
occurred within 10 second blocks of time while the children worked 
on the machines and these data were utilized to measure their levels of 
engagement and disengagement, 

The research staff met with three administrators, two teachers and 
an aide (who represented another teacher) during a Saturday morning 
session which was approximately three hours lon^ . The data were presented 
in the following orders 

CD Results of the Engagement Observation Scale — These data showed 
no significant differences in engagement or disengagement levels between 
the Choice and Non-choice groups* However t there were striking indivi- 
dual variations in these levels which occurred on a daily basis* 

(2) Choice of Difficulty Levala Data About one-half of the 
children in the first condition demonstrated two unusual types of choice 
patterns* ' The first pattern occurred when certain children continually 
selected easier problems although they were answering most of their 
previously worked problems correctly (Maximizers) , This pattern result- 
ed in their receiving arithmetic problems at grade levels which could 
not improve their academic n&rf ormance. The second pattern occurred 
when children selected more difficult problems although their previous 
performance was less than 65% correct (Minimizers) . The children in 
this sub-group performed poorly during most of their time on the computer, 



(3) Interview Data — ■ The children were asked why they decided 
to stop working on the computer and why they chose certain types of 
problems , These data indicated that many of the children stopped working 
because they wanted to complete certain classroom assignments. The 

most important interview finding was that the ninimizers 

said they chose ''harder*' problems because they wanted more "grown-up" 

types of problems , 

(4) Achievement Test Results — The primary goal of our research 
was to measure changes in student engagement rather than achievement . 
We reported these pre and post test results only to inform the May fair 
staff about the children's achievement levels during the time of the ex- 
periment. We did not expect differences in achievement to occur between 
our Choice and Non-choice conditions or between the children who worked 
and those who did not work on the computer* Our expectations* were con- 
firmed since there were no significant differences between these groups* 

■ The participants were asked to comment on the data after each one 

of the four presentations was completed* The information reported in thi 

paper is baaed upon the responses made during these periods and during 

a general discussion near the end of the session, A questionnaire was 

also administered (following the completion of the session) which asked 

about the significance of our research and the usefulness of the feed- 
2 

back session* Gome of the responses to this questionnaire are also re« 



"Appendix B contains a copy of this questionnaire* 



ported in this paper. However, it should be emohaaized that all of the 
topics which were discussed and commented upon are not described in this 
paper, Inst ad ? the author only included the topics which he believed 
would improve our relationships with May fair school and help us to in- 
terpret our results, 

Outcome of the Meeting 

Did the results of this meeting demonstrate that feedback sessions 
can be mutually advantageous to both researchers and teachers? First, 
the teachers offered helpful recommendatinns concerning how to improve 
some of our research procedures and interpret some of our findings. 
Secondly f we clarified soma of the curriculum options which can be used 
with the CM programs 3 and outlined sneeific steps that the teachers 
should take in order to more effectively use the CAI system. 

Procedures and Interpretations * Regarding the first point y there 
were three problem areas related to the experimental procedure which the 
teachers and administrators discussed with us ; 

(1) It was difficult to determine whether the children terminated 
their computer work because they were disengaged from the task or cer- 
tain pressures and attractions in the classroom ' demanded 1 ' that they 
leave the CAI room. The teachers confirmed our impression that par- 
ticular classroom activities caused some of the children to leave earlier 
than requirad and they advised us to modify our CAI schedule in a manner 
that would not conflict with classroom activities, This suggestion can 
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be easily implement 4 during this year v s research program, since the 
time for running children is more flexible than it was last year, 

(2) Our group achievement testing procedure seemed unreliable be- 
cause many of the children became restless and left the testing room 
before their allowed time elapsed. These tests were administered by 
members of our research staff or the observers outside of the children's 
classrooms. The Hay fair teachers recommended that we use the school 
achievement tests because their administration conditions were more con- 
stant than our conditions v and the teachers were more aware of vocabulary 
nroblems which might interfere with the children's understanding of the 
testing procedure. It was also suggested that the children understood 
our test results would not affect their academic progress. Therefore, 
their motivation to perform successfully was probably lower than usually 
expected, 

Clearly, teacher administered tests can reflect biases that yield 
unreliable and invalid scores, However, we are primarily interested in 
utilizing pre-test scores for sampling and matching purposes » and these 
achievement tests are probably less influenced by ! coaching*' s 'teaching 
for the test' 1 ' and other biasing factors at the beginning of the school 
year than near the end. In additions the teachers had more control 
over their pupils 1 test behavior than we did. Therefore, this recommen- 
dation will he followed by eliminating our achievement testing schedule, 

(3) We introduced an interview technique into our study in order 
to determine why the children stopped working on the computers why they 
made certain choices of problems and what they thought about working 
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arithmetic problems on the computer. However, many of their responses 
seemed to be more positive or less negative than expected, One adminis- 
trator stated that we might redesign our interview technique in order 
to determine whether the children refuse to ^ive negative responses 
because they are pressured by the interview situation and the phrasing 
of our questions. His proposal requires the use of negatively phrased 
statements such as, "The computer didn v t always give you what you asked 
for, M About one-half of the children would receive these statements 
while the other half would receive the standard interview questions. 
Comparisons between the frequencies of agreements and disagreements with 
the interviewer would indicate how intensively the children's explana- 
tions of their behavior were affected by pressures to agree with the in- 
terviewer. We have not followed this suggestion because our current 
interview questionnaire includes projective measures which will probably 
enable us to evaluate the effects of social influences upon the children's 
responses. However, two other suggestions made by an administrator 
and teacher will be implemented. They thought that we should ask the 
teachers about* the meanings of key words which children use in respond- 
ing to the interview questions # and the types of statements the children 
make about CAI upon returning from the computer room. 

The teachers' interpretations of our data are even more helpful 
than their procedural recommendations because they Indicate whether our 
findings are generaligable from the laboratory setting to the classroom. 
Thus, we wanted to know if the choice patterns which occurred on the com- 
puter are expected behaviors in other choice situations. If the teachers 
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were familiar with these choice patterns, then we asked for their ex- 
planations of why some children continually chose "harder problems and 
others continually chose ^easier ' ones. 

According to the teachers, these types of choice patterns can occur 
in the classroom and they appear to be more closely related to children ! s 
social status considerations than their academic performance. One 
teacher said that his pupils were more concerned with the grade level 
of their workbooks instead of the number of problems which they answered 
correctly ♦ In this regard, the children will ask each other whether they 
"have * a certain book rather than whether they can read the book effec= 
tively. 

What social pressures motivate children to select problems which 
are educationally inappropriate for them? The same teacher said that 
most of the children in our Study have experienced a long process of 
academic failure and were performing at about two grades below the fifth 
grade level. Therefore, they might derive social prestige from conspi- 
cuous indicators of academic success which can result in failure. For 
example 5 working In a book that is congruent with their grade level is 
desirable because they will at least fail in a ''socially prestigous" 
manner * 

Another factor which this teacher said would influence the child- 
ren 9 s choice patterns is related to his teaching objectives, Thus 9 
he stated that the children must first be taught to "work hard 11 since 
they will probably learn something if they develop this orientation to- 
wards work* The pressure to '-work hard H might have also influenced the 
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children who selected more difficult problems although they continually 
answered them incorrectly, Hence, the children in this group might 
have fulfilled their social prestige needs among both their peers and 
teachers by selecting these types of problems, Increased social pres- 
tige among their peers might have occurred because they were working 
problems at or above their grade levels (related to the first point dis- 
cussed by our teacher), In addition, the children might have obtained 
prestige from their teachers by r working hard' 1 on the computer, i.e, , 
selecting difficult problems, 

Why did certain children pick ''easier" problems which did not im- 
prove their grade level standings? The interpretations of this second 
choice pattern were not as extensive as those given for the first pattern 
and did not emphasize the social implications of making choices of 
""easier" problems, However 9 it was suggested that the children in this 
sub-group were from families 4 that did not develon a strong work ethic 
among their children, 

The teachers ? and administrators f explanations of different choice 
patterns reinforced our belief that certain social factors can influence 
CAI performance. Their descriptions of some of the social factors which 
affect classroom behavior indicated that we are investigating a general 
phenomenon which appears to have important influences upon different 
types of academic situations, Therefore, the discussion of this topic 
has caused us to focus our future research efforts uoon identifying some 
of the social influences (from parents* teachers and peers) which might 
affect children's decision-making behaviors. Furthermore, the teachers 1 
statements have also indicated that we should design methods for train- 
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ing children to make effective choices. In this regard, the application 
of our research findings to designing a training program would probably 
result in the more extensive application of our results to improving 
classroom instruction* ' 

Effective Use of the CAI Terminals. Our contract with Mayfair school 
specified that the teachers could run children on the terminals who 
were not involved in the study. The actual time allotted for general 
school use was between 8: 00-9! 00 A,M, and after 1-.00 P.M. 3 and the assist 
tant principal was responsible for scheduling and supervising children 
during these intervals. However, the teachers wanted to learn more 
about the types of programs and options which are available. For example, 
the Math, Drill and Practice program can be set at different grade 
levels according to a child f s abilities in about fourteen areas of 
arithmetic. It was concluded that the teachers should first decide how 
much time they want their children to devote to CAI and what types of 
exercises they would like them to receive. After this Initial planning, 
a meeting can be held with the computer programmers and the research 
staff in order to implement these plans* 



Results of the Questionnaire 

The teacher questionnaire was designed by Ruby Knowles who super- 
vises our program s s classroom studies of student engagement. The ques- 
tionnaire was administered to the school personnel who attended the feed- 
back session and two other teachers who participated in last year's study 
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but were unable to be at our meeting. The questionnaire concentrated 
upon three areas of interest* 

(1) Working Relationships between Gchool Staff and Researchers 

(2) Usefulness of the Project in On-going Teacher and Student Activities 

(3) Relevance of the Workshop Presentations to Classroom Teaching 

The data from this questionnaire will be reported in terms of the responses 
which provide us with new information about the amount of success we had 
in communicating the project 5 s goals and activities to the teachers* 

The first section indicated that the teachers obtained much of 
their information about the project from our observers, Although the 
observers were not trained to give such informations it appears that 
their informal contacts helped the teachers to understand the project 5 s 
goals and activities. Obviously 5 these contacts can become more syste- 
matic this year in order to improve the currant participants 4 understand- 
ing of our project. 

The teachers and administrators also stated that we should meet 
with them more frequently in order to effectively coordinate the CA1 study 
with the students' classroom work . For example s the assistant principal 
said that the research staff should n , , f be and feel like a part of our 
educational team at May fair," The need for more coordination between 
the CAI and classroom curricula was again emphasised in section two of 
the questionnaire (Usefulness of the Project in On-going School Activi- 
ties) 7 since two of the five individuals directly concerned with teaching 
were uncertain that M . . .what the students learned was consistent with 
their objectives and content for arithmetic in the classroom,- 4 In 
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addition, all of the teachers believed that the students who partici- 
pated in the CAI project were not more engaged in their arithmetic lessons 
in the classroom than the students who were unlnvolved in the research, 
However, these teachers demonstrated their interest in CAI by allowing 
fheir pupils to work on the teletypes when they were not being used 
by the researchers* 

The apparent interest of the teachers in CAI and their desire to 
work more closely with our staff suggest that future programming should 
concentrate upon the arithmetic topics which are being taught in the 
classroom, and either the program or teacher could w lead if the presenta- 
tion of these materials* The movement of our project in this direction 
requires a more sophisticated stage of reciprocity between researchers 
and teachers than presently exists. The first stage of our relation- 
ship was a trial-and-error period which concentrated upon setting up the 
technical apparatuses of Computer Assisted Instruction and introducing 
a suitable schedule for conducting this research. Following the resolu- 
tion of these prbblems, we then concentrated upon involving the school 
in using the terminals for non-experimental children who needed arith- 
metic drill exercises* Thirdly P the -feedback session moved us into a 
stage of reciprocity which enabled the teachers to evaluate the validity 
of our work* These evaluations appear to be positive and they suggest 
that a new stag© in our relationships should emerge in order to provide 
the teachers with useful research findings* Some of the activities 
in the fourth stage might include developing a program for training 
children to make effective choices and assisting teachers in utilizing 
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choice situations to imbrove academic performance. 

The last section of this questionnaire concentrated upon determin- 
in*j whether the meeting produced useful information for the teachers* 
Four out of five respondents believed that the most useful information 
was concerned with the choice data because the findings in this area 
of research can produce changes such as "improving student interest 
in learning H and producing more "individualized programs for children** 
However, only one teacher said that she would change some of her teach- 
ing methods as a result of the workshop, and none thought that the re- 
sults could definitely be applied to improving student interest and 
motivation* In contrasts all of the participants stated that! 
(1) educational research can produce useful knowledge s and (2) they 
would consider participating in future research projects conducted by 
our staff. 

Conclusion 

It appears that this feedback session Droduced two types of bene- 
ficial information which can have a positive influence upon our project 
future direction and is generalizable to other research projects in 
low-income schools. First, the teachers provided us with a "reality 
testing situation 1 ' for determining whether certain patterns of choices 
were expected in their classrooms* Their statements about the social 
determinants underlying these patterns verified some of our expectation 
concerning non-academic influences upon choice behaviors and suggested 
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that future research should identify the specific social factors which 
occur in different instructional situations* Secondly, the recommenda- 
tions related to promoting closer coordination between the CAI research 
and the classroom reveal that our project should move towards attaining 
an improved balance between research and development activities, The 
development aspect of our CAI work might emphasize two basic activities* 
(1) constructing programs 'viiich can be coordinated with the classroom 
curriculum^ and (2) designing instructional techniques which can improve 
children's decision-making skills* Clearly, such movements from the 
laboratory to the classroom also require the development of training 
programs that will enable teachers to effectively apply our research re- 
sults to their classrooms. It is expected that future teacher feedback 
sessions will produce helpful ideas concerning how to effectively apply 
the research on decision-making and engagement to developing teacher 
programs in low-income schools* 
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COMPONENT RESUME 
(Revised from Basic Program Plan, 3/31/72) 

Component Code: R1533B 

Stanford Center for Research 
Institution: and Development in Teaching Date Prepared: 1/24/73 

Program Title t Teaching Students from Low-Income Areas 

Component Title i Engaging Features of CAI Situations 



Start and end dates of proposed Component \ 9/7 1 - 11/73 



Staff member in charge I M. D, Fisher, R. D. Hess 



Problem , Educational settings which enable children to choose the 
difficulty levels of their curriculum may produce higher levels of motivation 
and engagement than settings which do not provide such choices. In this 
regard, no rigorous research information i s available concerning how low- 
income students 1 use of educational choices affects their levels of classroom 
engagement * 

Obj actives, Utilize Computer Assisted Instruction to: (1) Study how 
engagement is affected by giving children control over the difficulty levels 
of their arithmetic problems, (2) Identify sub-groups of children in terras 
of whether they make choices of problem difficulty levels that improve or 
interfere with their learning of CAI arithmetic problems, (3) Determine 
how children's sense of efficacy, as indicated by locus of control scores, 
is related to their engagement levels and the patterns of choices which they 
make. These patterns include children who either select: "harder 11 
problems although they rarely solve them correctly or "easier" problems 
which they seldom answer incorrectly, 

Strategy, (1) Design a computer program which will allow children to 
select the difficulty levels of their arithmetic problems. The curriculum 
for this program will be based upon Suppes f Math, Drill and Practice Exercises * 

ERLC 
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(2) Develop a locus of control instrument which measures children f s 
attitudes concerning how much control they have over their success in 
learning of arithmetic problems, (3) Place experimental (choice) and 
control (non-choice) groups on the computer for a period of' 15 days, 
(4) Administer the locus of control instrument to these children and record 
their levels of engagement-disengagement while they work on the machines. 

Projected outcomes, This study will provide information concerning how 
the choice of problem difficulty levels affects student engagement. It will 
also enable us to identify distinct patterns of choice behavior and to 
determine whether these patterns are related to locus of control and engage- 
ment scores. These findings can also provide teachers with specific choice 
patterns of different children and their reasons for making certain choices. 
This information can be utilized by teachers to identify children who must 
receive training in order to improve their learning in choice situations. 
Milestones . 

1. Completion of data collection for CM study — 5/73 

2. Completion of Technical Report on GA.X engagement study — 12/73 

3. Completion of dissemination activities to" researchers and 
teachers of CM data — 5/74 



Engaging Features of CAI Situations 



The CAI research component has been investigating some of the factors 
which we think are important in teaching, and that may stimulate the natural 
curiosity of children and help engage them in academic tasks. We have been 
operating two teletypes at Mayfalr Elementary School in San Jose, California, 
using the computer to simulate educational settings in which the pupils can 
exert control over their learning activities. The task of the study has been 
to evaluate the effects of these settings upon student engagement. 

Much has been written within the last ten years concerning! the child f s 
control over his learning destiny, and how this type of control can produce 
higher achievement levels in school and increased satisfaction with learning 
academic subjects. However, rigorous research has not been conducted 
upon how the opportunity to control the learning activities affectt: 
motivation. Does increased control over different learning activities 
produce increased motivation? This is the basic question that we are 
attempting to answer with our Computer Assisted instruction research. We 
think that the computer is one of the most rigorous instruments for investi- 
gating this issue. Therefore, the answers which are obtained from our 
research should yield clear-cut data about the effect of the student's 
control of his learning upon levels of engagement, 
1971-1972 Study 

Our initial research on the effects of choice upon student engagement 
began in February 1972 and will continue through May 1973, During this 
period we set up a computer project in a low-income school of San Jose and 
developed a computer program which would give children control over the 
difficulty levels of their arithmetic problems. Initially, we randomly 
selected similar numbers of boys and girls for two conditions i n) Control 



of Difficulty Levels (N=30) and b) No Control of Difficulty Levels 
(N*2i) . The subjects in both of these groups worked arithmetic problems 
for one half hour per day during a ten day period. At the end of each five 
minutes, the computer program asked them if they desired to stop or continue 
working problem;; . In addition, subjects offered "choice" were given a 
second question after every five minute period that asked them If they wanted 
"harder," "easier," or the "same" types of problems than they previously 
worked. The computer was programmed to maintain the "no choice" group at 
about a 70% correct level while the "choice" group started at approximately 
the 50% correct level (during the first five minute period) and could then 
select problems that were, either 15% harder or easier than those solved 
during the previous five minute period, 

Some of the data collected in this study were the? (1) observations of 
student engagement and disengagement,' and (?.) choices of difficulty levels 
made by subjects who received the second question. Figures 1 and 2 indicate 
the engagement and disengagement levels for the Control and No Control of 
Difficulty Levels groups, 
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figure?. 1, Engageinont Levels by Days, 
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The engagement and disengagement scores were based upon observations which 
were made during every ten second period while the children worked on the 
teletypes* The engagement scores were based upon such behaviors as reading 
problems silently } talking to self about problems } and counting on fingers. 
The points on the graphs represent the average number of occurrences of 
these behaviors per five minutes during each daily session* The graphs in 
Figure 2 were based upon indicators of disengagement. They included such 
behaviors as" turns away from the teietype ,f and"iooks away from the teletype," 
These data indicate that the two groups maintained similar levels of engagement 
during the ten day period, i.e,, there were no significant day-to-day 
differences between the groups and the decreases in engagement from days one 
through ten were similar. Although the "No Choice" group showed greater 
fluctuations in disengagement than did the "Choice" group, the differences 
between these groups was not significant* 

An analysis of the types of choices made by the Control of Difficulty 
Levels group showed a consistent pattern* Seventy-five percent of the 
choices for nine subjects appeared to maximiz e success; they were the 
"same" . or "easier" even though their percentage correct was between 85-100%* 
Another group of seven subjects ("Minimizers") appeared to minimize their 
performance; 87% of their choices were the "same" or "harder" when their 
percentage correct was between 0-64%. to help understand the basis for 
their choices we designed an interview questionnaire for the current year 
to determine why the subjects made certain types of choices* 

The results of the engagement and choice data for 1971-1972 have caused 
us to modify our techniques for studying student engagement and programming 
the computer. Thus, more dis tractors (such as toys and pictures) have been 
placed in the CAI room this year in order to provide the children with more 
variable stimuli than were present last year* Those new stimuli may produce 
more variability between the groups in this year f s study because they now 



have available some objects which can draw their attention away from the 
computer. The program this year has been modified so that the subjects 
now change the grade levels of the problems rather than their percentage 
correct. The classification of problems by grade levels has been developed 
by Suppes, and this change in our criterion for selecting problems has 
produced. almost 100% control over the choice of problems at various 
grade levels, Another important modification in our study for this year 
has occurred in our experimental design (which is described in the next 
section of this paper) . We have now attemped to control for the effects of 
working different sequences of problems by using a Yoked Control group which 
receives identical sequences as the Control of Choices group. 

Additional changes in our study for this year allow us to gather more 
extensive baseline engagement data than last year and to control the number 
of days which all children spend on the machines. 

1972-1973 Study 

This study includes two groups consisting of twenty subjects (equally 
divided between boys and girls) per group. All of the subjects will 
receive the Math, Drill and Practice Program and run for a maximum time of 
fifteen days at 35 minutes per day. The experimental conditions are as 
follows: 

1# Control of Difficulty Levels Grou p — The subjects in this group 
have been randomly selected and their starting points are at the first 
grade level. They will receive three questions (which have been written 
into the program) after they have worked on the teletypes for 15, 20, 25 
and 30 minutes. The questions are.: "Do you want more problems? Do you 
want harder, easier or the same types of problems Do you want a little, 
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a medium amount or a lot harder/easier problems?" (Level of difficulty is 
adjusted by -3, ,6 or ,9 of a grade level increments or decrements , ) 

After their first 15 minutes on the teletype during day one, the subjects 
can elect to continue the lesson or terminate. If they select the "don't 
want; to continue" option, the program stops working and they return to 
their classrooms . Questions two and three refer to the grade levels of the 
previously worked problems . The selection of "harder" or "easier" problems 
will change the grade levels of the problems which are presented during a 
subsequent time interval, However, the computer has been programmed to 
remain at constant grade levels within each time interval. 

After these subjects have completed working on the computer, they are 
interviewed to determine why they decided to stop working, and why they made 
the choices they did, The last question in this interview is concerned with 
whether they like to solve arithmetic problems on the computer* The primary 
purpose of this interview is to assist in interpreting data on the children's 
performance- on the computer. However, these questions do not yield the 
most rigorous types of data (for studying engagement and disengagement 
behavior) which result from this research. 

2. Yoked Control Group — The subjects in this group have been matched 
as closely as possible to particular subjects in Group I. The criteria 
utilized for matching are CA, sex of subject, achievement test score, grade, 
teacher and ethnic background. The starting points for these subjects are 
identical to that of the "choice" group. But they do_ not have options which 
allow them to stop working or to select the difficulty levels of their 
problems. In addition, their programs are set up to continue working so 
long as their cohorts ,/orked, and the difficulty levels of their problems 
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follow identical levels chosen by their cohorts , This control group 
enables us to control for the difficulty level sequences which the subjects 
in Group I select , since the subjects in Group II have identical sequences, 
Therefore, our comparisons will show whether the opportunity to choose 
problems of various difficulty results in higher levels of engagement and 
a greater sense of efficacy and control. This group will also be interviewed 
to determine why they think certain choice patterns can occur among different 
subjects^ and their preferences for working on the computer will be recorded. 
Measurement Instruments 

1* Engagem ent Observation Scale . — Observations are made during every 
10 second period while the children work on the teletypes. These data 
indicate various types of behavior which the children show during this time 
period, and these behaviors have been classified into engagement and 
disengagement categories, Some examples of these categories are ; 
(1) Engagement behavior — Eyes on teletype paper, talks to self about 
problems and pulls closer to the teletype; (2) Disengagement behavior — 
Turns away from the teletype, looks away from the teletype and yawns, 

2* Locu s of Control Test -- This test has been designed to measure 
whether the children believe they: (1) are responsible for their success 
or failure in learning arithmetic and (2) can control the computer program. 
Part I (general measure of Locus of Control) is given on day one before 
all the children begin working on the computer and Part II (specific LOG 
measure related to work on the computer) is given on days three and eight 
before they start working on the computer, Both Parts I and II are 
administered on the day after they have finished working the problems. 
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Research Questions which Compare t he Performance of the Control of 
Difficulty Levels (I) and Yoked Control Groups (II) 

1. Does the opportunity to choose various difficulty levels of 
„ arithmetic problems increase engagement and decrease disengagement? 

More specifically, does Group I show higher levels of engagement and 
lower levels of disengagement than Group II? Comparisons between these 
groups will be made during each of the fifteen daily sessions* If Group I 
has higher engagement and lower disengagement scores than Group II, we 
can conclude that giving children choices of difficulty levels yields 
higher levels of engagement than conditions, in which they have no choices, 
and the differences are not caused by changes or non-changes in the difficulty 
levels of the problems. 

Does the opportunity to choose difficulty levels of arithmetic problems 
encourage students to remain on the computer for similar amounts of time 
between days one and fifteen? This result might Indicate that the subjects 
have maintained similar engagement levels from days one through fifteen* 
However, the time data will be closely analyzed in relation to the engagement 
data, the reasons for stopping which are given by the children on each day 
and during their interviews . 

2- Does giving children the choice of difficulty levels affect their 
locus of control? 

More specifically, does Group I believe that learning outcomes are the 
result of more systematic sources of control than Group II? Does Group I 
attribute the locus of control more frequently to itself than does Gncft|p II? 
These questions will be answered by determining whether changes in the LOG 
Test scores occurred from the first to last administrations, and comparing 

O 
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the three groups in terms of differences in the amount of change, 
3. Do identifiable choice patterns emerge? 

The patterns which might be identified are: (1) Maximizers — Subjects 
who continually choose easier problems (maximize success) although their 
performance level is 90S correct or greater, and (2) Minimizes - Subjects 
who continually choose harder problems (minimize success) although their 
performance level is 50% correct or less. These sub-groups will also be 
eompared for possible differences between their mean engagement levels 
and locus of control, scores , 
Future Directions of the Project 

Although our computer program provides us with a rigorous technique 
for studying the effects of choice situations upon student engagement, it 
appears that research concerned with adults who are responsible for 
providing such choice situations for children will yield information which 
is also related to classroom behavior. Therefore, future research in this 
project might be concerned with the "controllers" (teachers, parents) of 
potential choice situations. One of the goals of this research might be 
to identify parents and teachers who allow relatively large an d small degrees 
of choice, and study the classroom behavior and achievement of their 
children. There appear to be other educationally significant choice 
situations which can be investigated by using the techniques developed in 
this program. For example, the effects of the voucher system and the use 
of student-teacher contracts might be studied to determine the conditions 
under which these types of choice situations increase student engagement. 
Our conversations with teachers and observations; of children working 
on the teletypes suggest that we should concentrate on the social correlates 



and educational consequences of choice. In regard to the first, we have 
observed children who will £se l ec t "easier" problems in order to produce 
the longest teletype printouts, and teachers have indicated to us that 
social-status factors influence their students' choices of textbooks (they 
want to be seen carrying a book which contains difficult assignments). 
It appears that these social-status factors wight have more important 
influences upon the children's choice behavior than academic and achievement 
factors. Investigations of the educational consequences of choice would 
be concerned with relating student engagement and achievement to various 
choice parameters . 
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Teacher's Experiences with the CM Project 



One of this project's alms is to design and implement studies that 
are mutually beneficial to school personnel mid researchers . This 
questionnaire attempts to gather information on the following aspects 
of teacher' s experiences with the CM project;: 

(1) Working Relationships between School Staff 
an d Res ear eh e rs 

(2) Usefulness of the project in on- going teacher 
and student activities 

(3) Relevance of the workshop presentations to 
classroom teaching 

Please help us to evaluate our efforts and to improve future 
research by responding to this questionnaire. Thank you for your 
cooperation in our work. 



Teachers 1 Experiences, with tho CAI Project 



Wgrking Re. In^^on^h iji^urt: w o q n Hoscarcii^r^mid_ School Staff 

1. Looking back on the contract upon which you and the research staff as- rood 
do you think it: (Chock one only) M C 

. covered all relevant details*, 



was incomp.lo.te., did not cover soma important details. 

Please specify which details, in your opinion, were not covered in 

the contract : 



Z, 



Looking back on the contract upon which you and the research staff oprced 
do you' think the terms were: (Check one only) ' 

. carried out as stated in the contract. 



violated in the course of the research. Please specify how the 
contract, in your opinion, was viol.it od : 



3, Looking at the project now, did you get enough information about it beforo 
you decided to participate? (Cheek all that apply) 

I would have lilted more information on 



I understand the project objectives differently now than at the ti 
I decided to participate. My original impression of the project 
objectives was 



me 



I got all the information 1 needed before my decision to participate. 
Did you have any contact with the observers from the CM research project? 



Yes 
No 



If you answered "yes" in Question 4 above, what was the nature of your 
contact with the observers? 

The observers were able to answer my questions about the research 
project, 

The observers came to the classroom to escort students to the computer 

Other, Please specify 
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L 1 !!"?!, 1 ! U V an3 f th ° u 8hts you may have on how the working relationship 
between the school and research* staff can be improved in feturo studies 
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XI , Use fulness of the Project, in On- gojjri j^ci i o c? 1 A ctivi tics 

At Relation of the CM Project to Teachers ' Activities 

7, Did you feel that your teaching was disrupted by the requirements 
of the research schedule? 

Yes , very much 

Yes , a little 

No 



If you answered "yes" in Question 7 5 please tell us in what ways 
your teaching war. disrupted 



9 8 If you answered "yes fi in Question 7 , please tell us how the disruption 
to your teaching can he minimized: 



10* Do you feel your students 1 learning in the classroom was disrupted by 
the requirements of the research schedule? 

_ Yes j very much 

m Yes, a little 

No 

11. If you answered "yes" in Question 10, do you have any ideas for 
minimizing the disruption experienced by your students? 



12. Do you think what your students learned with CAT was consistent with 
your objectives and content for arithmetic . in the classroom? 

_^ Yes 

m 1 am uncertain 

No 

13, If you answered "no" in Question 12 please indicate in what way (s) 
the CAI programs were inconsistent with your own teaching: 



14, Did you have the opportunity to work with the CAI arithmetic programs 
at the computer terminals? 



Yes 



No. (Please explain why you did not have the opportunity to 
work with the computer,) ___ 
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B. Rolatio n of the CAI Project to Student Activities 

15, During the hours which were not being used for the research 
project, d id your students use the computer? 

. Yes , I sent my students to the computer for CAT driij, 

Yc - S s m y students used the computer under the supervision of 

Mr. Sehuitz, 

No s my students did not use the computer outside of the 

research schedule. 

16, Do you think thai your students who participated in the CAI 
project were more engaged in their arithmetic Iuhsoiis in the 
classroom than your students who were not involved in the research? 
Yes . 

1 am uncertain. 
No. 

17 ■ Did you and your students talk about their experiences with the 
computer? 

_ Yes , 

No, 

If you answered !, yes n in Question 17 , which of the foil owing did yoi 
and students discuss? (Check all those that apply,) 

The observers from the research project, 

The content of the CAI programs, 

_ The operation of the computer terminals. 

___ The schedule for working with the computer, 

How the students felt about working with the computer, 
__ Other, ' (p] ease specify. ) 



18. Which of the following sentences best describes your students 1 feelings 
toward working with the computer? 

_ My students looked forward to working with the computer. 

My students were reluctant to work with the computer, 
I am not certain how my students felt. 

Other, (Please specify,) ____ 
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11 Ib jlrcsontntion of Find in Workshop 



19, 



20 



21 



Did today's workshop provide you with adequate Information 
regarding the objectives of this project? 



No 

Partly 
Yes 



If you answered "no" or "partly" in Question 19, were the 
presentations 

__ too technical 

1 n c o inp 1*2 1 u 
unclear 

too much Information was provided 



If you answered "no" or "partly" in Question 19, please indicate 
the kind of information you would have liked to receive: 



22. Do you think that the information provided in the workshop would 
be of interest to teachers whose students did not participate in 
this project? 



No 



P r ob ab ly not 
. I am uncertain 

Probably yes 

Definitely yes 



23, The Student KngaRcment instrument was described in thin mo rning f s 
workshop. Please check which of the following bolmvioru — in 3 
your judgment as a teacher ~ is characteristic of sLudont cngafienst-.iiL 
or disengagement in learning. (Please chuck only one category for 
each behavior.) " J 



Characteristic Not a 

Characteristic of both Characteristic 

Characteristic of Engagement and of Ffiiga^emcnt 

Behaviors of KngagetncnL Dig cnr;agei ncnt j^^gagament: or Disengagement 

Kycs on Paper 



Touches Keys 
of Terminal 

Pulls Chair 
Closer to 
Terminal 

Touches Paper 

Turns Away 



Reads Silently 

Talks, to Self 
Regarding the 
Problem 



Counts on 
Fingers 

Expresses 
Surprise 

Looks away 



24, Please describe any additional behaviors of engagements which you think 
should be added to the present instrument : 



25* Which of the following workshop presentations will be most useful to 
your classroom teaching in the future? 

(Please rank them 1-4 in order of usefulness,) 

Information on Observation Scale 
_ Information on Choice 

Information on the Interviews 
Information on Achievement Tests 
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Please describe in what ways Lhe information you ranked ] in Question 2b 
will be useful to your classroom teaching: 



27* As a result of today 's workshop, would you change some of the ways in 
which you tench your class? 

No 

Probably not 

__ 1 am uncertain 

Probably yea 

_____ Definitely yes 

28, If you answered "probably yes" or "definitely yes" in Question 27, 

please describe the ways in which you intend to change your teaching: 
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29 



30, 



In your opinion, can the research results reported today be an " ,\ 
by teachers to maintain student interest in classroom ins true ti _ n? 
No 

_____ Probably not 
______ I am uncertain 

Probably yes 

Definitely yes 



Do you think that educational research can produce knowledge which 
is useful for teachers and classroom instruction? 



No 

I am uncertain 
Yes 



31. Would you consider participating in future research projects conducted 
by this staff? 



Yes 



No * Please give the reasons why you would not participate in 

future research studies : 
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32. Please comment on nny a«pccu of the CAI project and thin morning's 
workshop which you think might help us improve (j.) our future 
research and (2) our relationships with school staffs and students . 
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